Brain imaging system and methods for direct prosthesis control

ABSTRACT

Methods and systems for controlling a prosthesis using a brain imager that images a localized portion of the brain are provided according to one embodiment of the invention. For example, the brain imager can provide motor cortex activation data using near infrared imaging techniques and EEG techniques among others. EEG and near infrared signals can be correlated with brain activity related to limbic control and may be provided to a neural network, for example, a fuzzy neural network that maps brain activity data to limbic control data. The limbic control data may then be used to control a prosthetic limb. Other embodiments of the invention include fiber optics that provide light to and receive light from the surface of the scalp through hair.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation-in-part, and claims the benefit, ofco-pending, commonly assigned U.S. patent application Ser. No.12/447,428, filed Aug. 3, 2009, entitled “Brain Imaging System andMethods for Direct Prostheses Control,” the entire disclosure of whichis incorporated herein by reference for all purposes.

STATEMENT AS TO RIGHTS TO INVENTIONS MADE UNDER FEDERALLY SPONSOREDRESEARCH OR DEVELOPMENT

The United States Federal Government may have rights to this inventionpursuant to National Science Foundation Grant Award Number CMMI-0729514.

BACKGROUND

The statistics for limb loss are sobering. Approximately 2 millionpeople in the United States suffer from limb loss. Each year more than185,000 amputations occur in the United States. It is estimated that oneout of every 200 people in the U.S. has had an amputation. Thestatistics for limb loss in developing countries are even moretroubling. Worldwide it is estimated that 650,000 people suffer fromupper-extremity limb loss.

Many prosthetic limbs are currently controlled by electromyography (EMG)and are referred to as myoelectric prostheses. Electromyography monitorsthe electric potential of flexor and extensor muscles in the remainingportion of the limb. Using the differential between the flexor andextensor muscles potential, it can be determined whether to close oropen a prosthetic hand. This system requires the user to consciouslyflex and relax muscles in order to control the artificial hand, becausethe activity of the remaining muscles would have normally controlled adifferent movement within the limb than the output of the prosthesis.

Other prostheses are actuated using mechanical and/or biosensors.Biosensors detect signals from the user's nervous or muscular systems,which is relayed to a controller located inside the device. Limbic andactuator feedback may be used as inputs to the function of thecontroller. Mechanical sensors process aspects affecting the device(e.g., limb position, applied force, load) and relay this information tothe biosensor or controller, for example force meters andaccelerometers. A prosthesis controller may be connected to the user'snervous and muscular systems as well as to the prosthesis itself. Thecontroller may send intention commands from the user to the actuators ofthe device, and may interpret feedback from the mechanical andbiosensors to the user.

Primary motor function of human muscles is directed within the motorcortex of the brain. The primary motor cortex is responsible for motionexecution and the premotor cortex is responsible for motor guidance ofmovement and control of proximal and trunk muscles. While sections ofthe motor cortex are relatively well mapped to muscles and/or musclegroups, understanding brain activity within such sections of the motorcortex is not well established. Previous attempts of brain imaging havetypically focused on large portions of the brain to map general zones ofthe brain to general functions.

BRIEF SUMMARY

A method for controlling a prosthesis is provided according to oneembodiment of the invention. The method includes receiving a pluralityof input signals from a brain imager, such as a NIR brain imager. Theinput signals may correspond to brain activity at one or more portionsof the motor cortex. A neural network, such as a fuzzy neural network,may then be used to map the plurality of input signals to a plurality ofoutput signals. The neural network may be trained to map an input signalassociated with the one or more portions of the motor cortex to anoutput signal that corresponds with one or more muscle groups The outputsignals may then be provided to a prosthesis, wherein the prosthesis isconfigured to respond to the output signals. The method may also includeilluminating light on a portion of the brain using one or more lightsources and receiving a plurality of light signals at the surface of thescalp using a plurality of photodiodes. The light sources may includeLEDs, fiber optics and/or lasers. Detected light may have traveled fromthe one or more light sources through a plurality of sub-portions of thebrain and may be detected at the plurality of photodiodes. This detectedlight may then be provided as a plurality of input signals.

A prosthesis control system that includes a brain imager is providedaccording to another embodiment of the invention. The system includesone or more light sources, a plurality of photodiodes, a controller anda prosthesis. The one or more light sources are configured to irradiatelight into a first portion of the brain, such as the motor cortex. Thelight sources may include LEDs, lasers and/or fiber optics. The lightmay be near infrared light. The plurality of photodiodes may beconfigured to detect a portion of the light transmitted into the firstportion of the brain. The photodiodes may receive light from the scalpthrough a fiber optic. The detected light may travel at least from theone or more light sources through a plurality of sub-portions of thebrain and be detected at the plurality of photodiodes. The controllermay be configured to receive a plurality of inputs from the plurality ofphotodiodes. The controller may perform a plurality of functions. Forexample, the controller may determine the relative concentration ofoxy-hemoglobin and/or deoxy-hemoglobin within the first portion of thebrain from the plurality of photodiode inputs. The controller maydetermine the brain activity at a plurality of sub-portions of the firstportion of the brain from the relative concentrations of oxy-hemoglobinand hemoglobin. The controller may also determine a plurality of limbiccontrol signals from the brain activity within the first portion of thebrain. The prosthesis may be configured to receive the limbic controlsignals from the controller and configured to operate in response to thelimbic control signals. The controller may include a neural networkinference engine that is configured to determine the plurality of limbiccontrol signals from the brain activity within a first portion of thebrain. The system may also include a headset with photodiodes, fiberoptics and/or light sources embedded therein.

A fiber optic for transmitting light into the brain through the scalpand past hair is provided according to another embodiment of theinvention. The fiber optic includes an optical fiber and a bulb. Theoptical fiber includes a distal end, a proximal end and elongated fiberbody. The proximal end is configured to receive light from a lightsource and the elongated fiber body is configured to channel lightreceived from the light source at the proximal end to the distal end.The bulb is coupled with the distal end of the optical fiber. The bulbis configured to transmit light from the elongated fiber optic body intothe brain through the scalp and past hair. The bulb may be substantiallyspherically shaped, that is, the bulb may be a sphere, spheroid,hemispherical, oblong, oval, etc. The bulb may also comprise ahemisphere. The optical fiber and bulb may comprise the same materialand may be fused together.

A method for training a prosthesis system is disclosed according toanother embodiment of the invention. The prosthesis system may include aneural network, a brain imaging system and a prosthesis. The trainingmay utilize an electromyograph. The method may include receiving brainactivity data from the brain imaging system, receiving muscular responsedata from the electromyograph, wherein the muscular response datacorresponds with the brain activity, and training the neural network toproduce the muscular response data from the brain activity data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a homunculus of the human brain showing the relative areasand size of portions of the motor cortex that are used for limbiccontrol.

FIG. 2A shows a brain imaging headset according to one embodiment of theinvention.

FIG. 2B shows how a brain imaging headset may be repositioned on thescalp according to one embodiment of the invention.

FIGS. 3A, 3B and 3C show configurations of light sources and lightdetectors for use with a brain imager according to one embodiment of theinvention.

FIG. 4 shows the specific absorption of water versus wavelength.

FIG. 5 shows the specific absorption of hemoglobin and oxy-hemoglobinversus wavelength.

FIG. 6 shows a cut-away of a human skull showing the brain, skull andbrain imaging sensors according to one embodiment of the invention.

FIG. 7 shows a block diagram of a brain imaging sensor according to oneembodiment of the invention.

FIG. 8 shows a fiber optic connector according to one embodiment of theinvention.

FIG. 9 shows a graph of the averaged levels per channel obtained usingone embodiment of the brain imager of the present disclosure.

FIG. 10 shows a block diagram of a multilayered fuzzy neural networkaccording to one embodiment of the invention.

FIG. 11 shows a block diagram of a prosthesis control system using abrain imager according to one embodiment of the invention.

FIG. 12 shows a block diagram of a system for controlling a prosthesisusing a brain imager according to one embodiment of the invention.

FIG. 13 shows a high level view of a prosthetic control system using abrain imager according to one embodiment of the invention.

FIG. 14 shows a flowchart showing a method for using a brain imager fordetermining brain activity related to a portion of the motor cortex thatis translated into actuation of a prosthesis according to one embodimentof the invention.

FIGS. 15A and 15B show two exemplary ends of a fiber optic according toone embodiment of the invention.

FIG. 16 shows a system that combines EEG signals, EMG signals and nearIR signals for adaptive artificial limbic control according to someembodiments of the invention

FIG. 17 shows another high-level block diagram of a prosthesis controlsystem according to some embodiments of the invention.

FIG. 18 shows a flow chart of process for training a combined near IRand EEG system according to some embodiments of the invention.

FIG. 19 is an example of process that can be used by the controller forusing a combined EEG and IR brain imager to actuate a prosthesisaccording to some embodiments of the invention.

FIG. 20 shows a flow chart of a process for using a brain imaging systemfor lie detection according to some embodiments of the invention.

DETAILED DESCRIPTION OF THE INVENTION

The ensuing description provides exemplary embodiment(s) only, and isnot intended to limit the scope, applicability or configuration of thedisclosure. Rather, the ensuing description of the exemplaryembodiment(s) will provide those skilled in the art with an enablingdescription for implementing an exemplary embodiment. It beingunderstood that various changes may be made in the function andarrangement of elements without departing from the spirit and scope asset forth in the appended claims.

One embodiment of the invention provides for various systems and methodsfor providing prosthetic control using a brain imager. For example,according to one embodiment of the invention, a near infrared (NIR)brain imager is configured to detect motor cortex activationspecifically at a portion of the motor cortex corresponding with aspecific muscle or muscle group. This motor cortex activation data maythen be translated into limbic control signals using a neural network,for example, a fuzzy neural network, and then used to actuate anartificial limb. A fuzzy neural network is provided that quickly learnslimbic actuation outputs from brain activation data according to anotherembodiment of the invention. As another example, a fiber optic isprovided that may be used to transmit light into and/or receive lightfrom the brain. Various other embodiments will be described throughoutthe body of this disclosure.

This disclosure provides a description of various embodiments of theinvention and is organized as follows: First, a brain imager thatprovides noninvasive localized brain activity data is describedaccording to one embodiment of the invention is disclosed. A specificexample of such a brain imager, a NIR brain imager, is then presentedaccording to one embodiment of the invention along with a description ofits operation using the Beer-Lambert Law. A neural-fuzzy inferenceengine is described, according to another embodiment of the invention,that provides learned limbic outputs from brain activity inputs. Atraining system according to another embodiment of the invention is thendescribed that may be used to associate specific brain activity withspecific limbic outputs. Fiber optic sensor and/or detectors aredisclosed according to another embodiment of the invention. Finally, asystem that employs a brain imager and neural network to control anartificial limb is disclosed.

I. Brain Imager

There is a precise somatotopic representation of the different bodyparts in the primary motor cortex, as shown by the motor homunculus inFIG. 1. The arm and hand motor cortex is the largest, and lies betweenthe leg and face area. The lateral area of the primary motor cortex isarranged from top to bottom in areas that correspond to the buttocks,torso, shoulders, elbows, wrists, fingers, thumbs, eyelids, lips andjaw. Interior sections of the motor area folding into the mediallongitudinal fissure correspond with the legs. Different portions of themotor cortex are activated during control of specific portions of thebody.

A brain imager configured to provide noninvasive brain activity datafrom a select portion of the brain is provided according to oneembodiment of the invention. The brain imager, for example, may utilizelight to detect indications of brain activity within a select portion ofthe motor cortex. This brain activity may correspond with a specificmotor function. For example, the brain imager may only monitor theportion of the brain that is used to control, for example, the forearm,ankle, shoulder, wrist, leg, hand, hip, or foot, etc. In one embodimentof the invention, the brain imager provides localized brain activitydetection over only the specific portion of the brain that is associatedwith a specific motor function.

The brain imager may include a number of sensors and/or sources. Thesensors and/or sources may be located on an adjustable headset 200 asshown in FIGS. 2A and 2B. The sensors may include optical sources,magnetic sources, photodiodes, magnetic sensors, electrical sensors,etc. The headset 200 may be adjustable such that the sensors and/orsources may be moved to sense different portions of the brain. Moreover,the sensors and/or sources may be localized by the adjustable headset ona specific portion of the scalp above the portion of the motor cortexthat is of interest. Moreover, the sensors and/or sources may beconfigured in a dense pattern within a detector-source array.

FIGS. 3A, 3B and 3C show various examples of source 340 and sensor 350configurations according to embodiments of the invention. As shown, thesources 340 and sensors 350 are configured on a sensor-detector array310. The sensor-detector array 310 may be configured to include one ormore sensors 350 and/or sources 340 in localized pattern near anobservable portion of the brain. For example, the sensor-detector arraymay be placed on a user's scalp with the array 310 located over theforearm portion of the motor cortex in order to image the brain activityof the forearm portion of the motor cortex. In another example, thesensor-detector array may be placed on a user's head with the array 310located over the ankle portion of the motor cortex in order to image thebrain activity of the ankle portion of the motor cortex.

A brain imager that focuses on a specific portion of the motor cortexdoes not image the entire brain or the entire motor cortex. Instead, thebrain imager, according to embodiments of the invention, may provide ahigh density of sources and/or sensors on the sensor-detector array 310.Accordingly, because of the density of sensors over a specific portionof the motor cortex, the brain imager provides a plurality of brainactivation signals for a specific motor function. Thus, the brain imagerprovides greater activation resolution for a specific brain function,for example, specific motor cortex activity.

The brain imager may include a near infrared brain imager, a magneticresonance brain imager, an electromagnetic brain imager, etc.

A. NIR Brain Imager

A near infrared (NIR) brain imager may be used as a specific type ofbrain imager according to one embodiment of the invention. FIG. 4 showsa graph of the specific absorption coefficient of water versus thewavelength of incident light. As shown, the absorption increasesdramatically for light above 820 nm. The absorption of light is lowenough in the near infrared (NIR) range that a substantial portion oflight in this range is transmitted through water without absorption.Because the human body, including the brain, is comprised mostly ofwater, NIR light transmits well through the brain.

FIG. 5 shows the specific absorption of hemoglobin and oxy-hemoglobinversus wavelength in the near infrared. As shown, oxy-hemoglobin andhemoglobin (deoxy-hemoglobin) produce a unique spectral signature withinthe near infrared spectra. When light passes through tissue, the photonsundergo multiple reflections due to scattering objects in their paths,and in the process some of the photons get absorbed by chromophoresand/or by other absorbing substances in the tissue, before the lightemerges out of the tissue. Accordingly, when light passes throughtissue, the tissue imposes its optical signature on the some of thelight. By making a comparison between the properties of the light beforeentering the tissue and after it emerges from the tissue, the opticalsignature of the tissue can be determined.

Light propagation in tissue is governed by photon scattering andabsorption. The overall effect of absorption is a reduction in the lightintensity traversing the tissue. The relationship between the absorptionof light in a purely absorbing medium and the structure and pigmentspresent in the medium is given by the Beer-Lambert Law. Scattering isthe basic physical process by which light interacts with matter. Changesin internal light distribution, polarization, and reflection can beattributed to the scattering processes. Because scattering increases theoptical path length of light propagation, photons spend more time in thetissue when no scattering occurs thus changing the absorptioncharacteristics of the medium. Light propagation in a turbid(scattering) medium can be modeled by the modified Beer-Lambert Law(MBLL).

The electromagnetic spectrum has two unique characteristics in the NIRrange (700 nm-1000 nm). First, biological tissues weakly absorb NIRlight, allowing it to penetrate several centimeters through the tissueand still can be detected. In addition, the dominant chromophores (lightabsorbing molecules) in the NIR window are oxy-hemoglobin anddeoxy-hemoglobin. The principal chromophores in tissue are water(typically 80% in brain tissue), lipids, melanin and hemoglobin. Theabsorption spectrum of lipids closely follows that of water and melaninthough an effective absorber contributes only a constant attenuation.Spectroscopic interrogation of tissue reveals that oxy-hemoglobin(oxy-Hb) and deoxy-hemoglobin (doxy-Hb) are biologically relevantmarkers, and their neurovascular coupling allows absorption spectra toreliably track neural activity. There are at least three types of NIRimaging: (1) time resolved (TRS), (2) frequency domain (FD), and (3)continuous wave (CW) techniques.

1. Beer-Lambert Law (BLL)

The Beer-Lambert Law is an empirical relationship that maps absorptionof light to the properties of the material through which the light istraveling. There are several ways in which the law can be expressed. Thetransmittance (T) of light through a medium, which is the ratio of theintensity of light that enters a medium (I_(O)) over the intensity ofthe light that exits the medium (I_(I)) may be expressed as:

$\begin{matrix}{{T = {\frac{I_{o}}{I_{I}} = ^{{- \alpha} \cdot l \cdot c}}}{where}} & {{eq}.\mspace{14mu} 1} \\{\alpha = {\frac{4\pi \; k}{\lambda}.}} & {{eq}.\mspace{14mu} 2}\end{matrix}$

In terms of absorbance (A) of light

$\begin{matrix}{A = {\alpha \cdot l \cdot c}} & {{eq}.\mspace{14mu} 3} \\{A = {\log {\frac{I_{o}}{I_{I}}.}}} & {{eq}.\mspace{14mu} 4}\end{matrix}$

where l is the distance that the light travels through the material (thepath length), c is the concentration of absorbing species in thematerial, α is the absorption coefficient or the molar absorptivity ofthe medium, λ is the wavelength of the light, and k is the extinctioncoefficient.

Accordingly, the Beer-Lambert Law states that there is an exponentialdependence between the transmission of light through a substance and theconcentration of the substance, and also between the transmission andthe length of material that the light travels through. Thus, if l and αare known, the concentration of a substance can be deduced from theamount of light transmitted by it.

2. Modified Beer-Lambert Law (MBLL)

An example of the geometry of NIR light propagation through the brain isshown in FIG. 6 and depicted schematically in FIG. 7. In FIG. 6, aheadband 600 positions a light source 610 on the skull 620 of the userto direct input light at an intensity (I_(I)) to a target tissue area630 of the brain 640. One light source and two detectors are shown forsimplicity and by way of example. An output light intensity (I_(o)) isreceived by a light detector 650. From the light source 610, lightfollows a characteristic path 460 through the target tissue 630 back tothe light detector 650 on the same approximate plane 670 as the source610. While the light is severely attenuated by the intermediate tissue680 (including hair, skin, and bone tissue such as found on a person'shead) and by the target tissue 630 due to the scattering and absorptionprocess, it is nonetheless encoded with the spectroscopic signatures ofthe molecules encountered en route to the light detector 650.

Looking at FIG. 7, fractions of the incident light that are remitted,scattered and absorbed depend on the optical properties of the targettissue 730. The amount of absorption is directly dependent on thechromophore concentration. The optical path 760 taken by remittedphotons is a characteristic “crescent shape” whose dimensions,particularly the depth of penetration L, are dictated by thesource-detector separation d.

FIG. 7 shows NIR source-detector geometry schematically according to oneembodiment of the invention. NIR recordings are basically quantifiedtrend measurements. They do not attempt to predict the absolute oxygenlevel at any given time, but track neural activity by recording theoxygen level changes with time. By applying MBLL to the source-detectorgeometry, the following is obtained:

$\begin{matrix}{A_{\lambda} = {{\log_{10}\left( \frac{I_{0}}{I_{I}} \right)} \approx {{ɛ_{\lambda} \cdot c \cdot d \cdot {DPF}} + G}}} & {{eq}.\mspace{14mu} 5}\end{matrix}$

Where A_(λ) is the light intensity attenuation for wave length λexpressed in terms of optical density (OD). 1OD corresponds to a 10 foldreduction in intensity. I_(I) and I₀ are the input and output lightintensities respectively. ε_(λ) is the absorption factor for wavelengthλ. ε_(λ) is also called the specific absorption coefficient or theextinction coefficient for wavelength λ. ε_(λ) is defined as the numbersof ODs of attenuation produced by the absorber at a concentration of 1μM (micro moles) and over a physical path of 1 cm, hence the dimensionsof OD are cm⁻¹ μM⁻¹. c is the concentration of the chromophore in termsof μM. d is the distance between the source and detector in terms of cm.DPF is the differential pathlength factor, which is a dimension lessconstant to account for the photon path lengthening effect of scatteringand G is an additive term for fixed scattering losses.

Eq. 5 can be rewritten as:

$\begin{matrix}{{\Delta \; A_{\lambda}} = {{\log_{10}\left( \frac{I_{0}(t)}{I_{0}(0)} \right)} = {\Delta \; {c \cdot ɛ_{\lambda} \cdot d \cdot {DPF}}}}} & {{eq}.\mspace{14mu} 6}\end{matrix}$

The two chromophores oxy- and deoxy-hemoglobin can then be taken intoaccount by:

$\begin{matrix}{{\Delta \; A_{\lambda}} = {{\log_{10}\left( \frac{I_{0}(t)}{I_{0}(0)} \right)} = {\left( {\sum\limits_{t = 1}^{2}{\Delta \; {c_{i} \cdot ɛ_{\lambda \; t}}}} \right) \cdot d \cdot {DPF}}}} & {{eq}.\mspace{14mu} 7}\end{matrix}$

A similar measurement at another wavelength is needed to solve for thetwo Δc, turning eq. 7 into a matrix-vector:

$\begin{matrix}{\underset{A}{\underset{}{\begin{bmatrix}{\Delta \; A_{\lambda \; 1}} \\{\Delta \; A_{\lambda 2}}\end{bmatrix}}} = {\frac{1}{d}\underset{ɛ}{\underset{}{\begin{bmatrix}\frac{ɛ_{\lambda \; {1 \cdot {oxyHb}}}}{{DPF}_{\lambda \; 1}} & \frac{ɛ_{\lambda \; {1 \cdot {doxyHb}}}}{{DPF}_{\lambda \; 1}} \\\frac{ɛ_{\lambda \; {2 \cdot {oxyHb}}}}{{DPF}_{\lambda \; 2}} & \frac{ɛ_{\lambda \; {2 \cdot {doxyHb}}}}{{DPF}_{\lambda \; 2}}\end{bmatrix}}}\underset{C}{\underset{}{\begin{bmatrix}{\Delta \; c_{oxyHb}} \\{\Delta \; c_{doxyHb}}\end{bmatrix}}}}} & {{eq}.\mspace{14mu} 8}\end{matrix}$

Careful selection of the wavelengths will result in a nonsingular callowing solution by direct matrix inversion. The final two measures ofoxygenation (oxy), and blood volume (BV), are extracted from the Δc_(i)as:

oxy=ΔC _(oxyHb) −ΔC _(doxyHb)  eq. 9

BV=ΔC _(oxyHb) +ΔC _(doxyHb)  eq. 10

Dimensions of both oxy and BV are in μM. An accurate value of DPFaccounting for its dependence on wavelength is give by:

$\begin{matrix}{{DPF}_{\lambda} = {\frac{1}{2}{{\left( \frac{3\mu_{s\; \lambda}^{\prime}}{\mu_{a\; \lambda}} \right)^{\frac{1}{2}}\left\lbrack {1 - \frac{1}{1 + {d\left( {3\mu_{s\; \lambda}^{\prime}\mu_{a\; \lambda}} \right)}^{\frac{1}{2}}}} \right\rbrack}.}}} & {{eq}.\mspace{14mu} 11}\end{matrix}$

where μ′_(sλ) is the reduced scattering coefficient of blood atwavelength λ and μ_(aλ) is the absorption coefficient of blood atwavelength λ.

The depth of light penetration (L) in FIG. 7 plays a role in theinformation content of the extracted signal. Optical neural-imaginginvolves light propagation through layers of heterogeneous tissue withcerebrospinal fluid (CSF) influencing the depth of light penetration.

B. Exemplary Brain Imager

With the analytical framework discussed above in regard to the BLL andMBLL in mind, a NIR brain imager may provide relative oxy anddeoxy-hemoglobin measurements from the intensity of light transmittedinto a portion of the brain (I_(I)) and the intensity of light receivedfrom the portion of the brain (I_(o)). One embodiment of the inventionuses coherent light sources, such as LEDs or lasers, with peakfrequencies centered at 735 nm and 850 nm. The brain imager may alsoinclude one or more light detectors, such as photodiodes, that detectlight transmitted into the brain by the light source(s). For example, asingle light source may be surrounded by an array of six photodiodes at735 nm and an array of six diodes at 850 nm within a singlesource/detector package. Other examples may include a light source withthree photodiodes at 735 nm and 850 nm. In yet another example, adetector at 735 nm and 850 nm may be surrounded by four light sources.Such configurations may generate enough power to provide a sufficientsignal back from the brain. Another embodiment uses LEDs with a singlefilament at each wavelength and a smaller distance between source andreceiver to compensate for signal loss. The two wavelengths being usedcannot be adjusted inasmuch as these wavelengths correspond with thespectrally significant portions of the signal received back from thebrain. Various source/detector configurations may be used. In someembodiments, multiple light sources may be included with a singledetector and vice versa.

FIGS. 2A and 2B show examples of an optical brain imager 200 accordingto one embodiment of the invention. As shown, a headset is shown with anumber of fiber optic cables 240 that transmit light to and from thesurface of the scalp. It is to be understood that any other suitabledevice could be utilized to incorporate the brain imager such as aheadband, a cap, a support fixture, etc. The headset 200 of FIG. 2Bpositions the array 210 above the head 220 of the user 230 a givendistance so that the fiber optics 240 extend downwardly from the array210 through the hair (not shown) to touch the scalp. The headset 200 canbe placed as close to the scalp as possible. The distance that the fiberoptics 240 extend below is dependent on how close the headset 200 is tothe scalp. One end of the fiber optics 240 is positioned as close to thescalp as possible and the other end can be coupled with a light sourceor light detector to transmit or receive light.

C. Sensor-Detector Array

The array 310, as shown in FIG. 3A, is a 6×24 array, that is six lightsources 340 are surrounded by four light detectors 350. As shown inFIGS. 2A and 2B, in one embodiment of the invention, thin optical fibersshine light from light sources between hair (not shown) to the scalp,and fiber optics can detect neural-imaging signals and return the lightfrom the surface of the scalp to a light detector. The fiber optics arecoupled with the light sources and the detectors. The light sources anddetectors are located in a separate unit that is shoulder orback-mounted with no electrical wires being run to the headset. Thelight is brought up to the headset and back down from the headset viaoptical fibers. Accordingly, fiber optics may be considered part of thelight sources and/or part of the light detectors.

As shown inn FIG. 3A, the spacing of the sensors may be 1.41 inches fromcenter to center in a uniform pattern both across and along the lengthof the device. This spacing (square-root of 2) places the light sources1.00 inch from all surrounding light detectors (center to center) 350.Various other dimensions may also be used. FIG. 3B shows light detectors350 surrounded by a plurality of light sources 340 in another geometricpattern. The sensors and/or light sources may include a fiber optic.FIG. 3C shows another array with each light detector 350 is surroundedby three light sources 340. Various other sensor and light sourceconfigurations may be used without deviating from the spirit of theinvention. Light detectors 350 and light sources 340 may be about 0.5,1.0, 1.5, 2.0, 2.5, 3.0, 3.5, etc. centimeters apart. Moreover, thesensors 350 and light sources 340 may be configured in any pattern andmay be positioned away from the array but coupled thereto with fiberoptics.

The array may present light sources and/or detectors at portions of thescalp corresponding to the pre-motor cortex and/or the motor cortex.

In one embodiment of the invention, the sensor-detector array mayinclude a plurality of fiber optics. Each fiber optic may be associatedwith a light source or a photodiode at either a first or secondwavelength. The fiber optics may be arranged in any of a number ofpatterns. The density of the fiber optics may also vary. Thesensor-detector array provides light and receives light from a specificportion of the scalp.

FIG. 8 shows a fiber optic connection apparatus 880 according to oneembodiment of the invention. The apparatus includes a housing 882 forholding an LED 884, a ball lens 886, and a fiber 888 for carrying lightemitted from the LED 884 through the ball lens 886 to the end of thefiber optic.

FIG. 9 shows a graph of the relative levels of blood volume, oxygen,hemoglobin, and oxy-hemoglobin from a system employing variousembodiments of the invention. Ten photodiodes and four light sourceswere used to create a 16-channel detector. The optical brain imager wasused on six human subjects. These subjects were asked to perform somemovements while the photodiode outputs were being collected. During aseated leg lift, the general tendency was for oxygenated blood volume todecrease and deoxygenated blood volume to increase in areas of the brainthat were activated by the movement. The odd numbered channelsrepresented the front row of detectors and even numbered channelsrepresented the back row of detectors.

In another embodiment of the invention, the array includes a 20×80array. Such an array may obtain more data points with a higher sensordensity or a larger monitoring area. Other array configurations may beused, for example, arrays with 10, 20, 30, 40, 50, 60, 70, or 80 lightdetectors and arrays with 10, 20, 30, 40, 50, 60, 70, or 80 lightsources may be used. Any other combination of light sources and/or lightdetectors may be used.

In another embodiment of the invention, a plurality of sensor arrays areprovided. Each of the sensor arrays may be used to sense brain activityat a different specific portion of the brain. For example, a headset maybe coupled with two sensor arrays. One sensor array may be positionedover the wrist control portion of the motor cortex and the other sensorarray may be positioned over the elbow control portion of the motorcortex. Thus, activation of brain activity in either the wrist and/orthe elbow motor cortices will provide signals to a controller that maybe used to control the wrist and/or elbow. Any number and/orcombinations of motor cortex arrays may be imaged. In yet anotherembodiment of the invention, a headset may include a plurality ofarrays, where each array includes a small number of densely packed lightsensors and/or light detectors. For example, a head set may include fourarrays with each array containing six light detectors and six lightsources.

II. Neural-Fuzzy Inference Engine

A neural-fuzzy inference engine is also provided for mapping brainactivity data into limbic control signals according to one embodiment ofthe invention. Mapping brain activity to limbic control can be seen asan inverse nonlinear problem with some level of uncertainty due to thefinite resolution of optical brain imager. A mix of neural network andfuzzy logic may be incorporated in the inference engine. While variousinference engines, if/then engines, neural networks or the like may beused to map brain activity data to limbic control, a neural-fuzzyinference engine is provided as one example.

A neural-fuzzy inference engine may have five layers, in one embodiment,and can be used for any number of multi-inputs and multi-outputs (MIMO).The neural-fuzzy inference engine employs the gradient descent methodand the least square estimation (LSE) algorithms to train the network.FIG. 10 shows the architecture of the inference engine.

Layer 1 (L₁) is a fuzzification layer. Each node generates a membershipdegree of a linguistic value. The k^(th) node in this layer performs thefollowing operation:

$\begin{matrix}{O_{l}^{1} = {{O_{ij}^{1}\left( x_{i} \right)} = \frac{1}{1 + \left( \frac{x_{i} - a_{ij}}{b_{ij}} \right)^{2}}}} & {{eq}.\mspace{14mu} 12}\end{matrix}$

where j is the number of membership functions, i is the number of inputvariables, l=(i−1)·n_(i)+j and x_(i) is the i^(th) input variable. Theantecedent parameters {a_(ij),b_(ij)} are a set of parameters associatedwith the j^(th) membership function of the i^(th) input variable andused to adjust the shape of the membership function during training.

Layer 2 (L₂) is a multiplication layer. At the multiplication layer,each node calculates the firing strength of each rule by usingmultiplication operation.

$\begin{matrix}{{O_{k}^{2} = {\prod\limits_{i}{O_{ij}^{1}\left( x_{i} \right)}}}{\left( {1 \leq k \leq 4} \right),}} & {{eq}.\mspace{14mu} 13}\end{matrix}$

where k is an integer between 1 and the number of nodes in the secondlayer and O_(k) ² is the output of the k^(th) node in the second layer.

Layer 3 (L₃) is the normalization layer. The number of nodes in thislayer may be the same as the first layer. The output of layer 3 isdetermined according to:

$\begin{matrix}{{O_{k}^{3} = \frac{O_{k}^{2}}{\sum\limits_{k}O_{k}^{2}}}{\left( {1 \leq k \leq 4} \right).}} & {{eq}.\mspace{14mu} 14}\end{matrix}$

Layer 4 (L₄) is the defuzzification layer. The number of nodes in thislayer may be equal to the number of nodes in layer 1 times the number ofoutput variables. The defuzzified value for the k^(th) rule is

$\begin{matrix}{{y_{k} = \begin{Bmatrix}{c_{k} - {d_{k}\sqrt{\frac{1}{O_{k}^{3}} - 1}}} & {{{if}\mspace{14mu} k} = {odd}} \\{c_{k} + {d_{k}\sqrt{\frac{1}{O_{k}^{3}} - 1}}} & {{{if}\mspace{14mu} k} = {even}}\end{Bmatrix}}\left( {1 \leq k \leq 4} \right)} & {{eq}.\mspace{14mu} 15}\end{matrix}$

where {c_(k), d_(k)} are consequent parameters and are used to adjustthe shape of the membership function of the consequent part. Then, theoutput of layer 4 becomes:

$\begin{matrix}{\begin{matrix}{O_{k}^{4} = {O_{k}^{3} \cdot y_{k}}} \\{= \begin{Bmatrix}{O_{k}^{3} \cdot \left( {c_{k} - {d_{k}\sqrt{\frac{1}{O_{k}^{3}} - 1}}} \right)} & {{{if}\mspace{14mu} k} = {odd}} \\{O_{k}^{3} \cdot \left( {c_{k} + {d_{k}\sqrt{\frac{1}{O_{k}^{3}} - 1}}} \right)} & {{{if}\mspace{14mu} k} = {even}}\end{Bmatrix}}\end{matrix}{\left( {1 \leq k \leq 4} \right).}} & {{eq}.\mspace{14mu} 16}\end{matrix}$

Layer 5 is the summation layer. Here, the number of nodes is equal tothe number of outputs. There is only one connection between each node inlayer 3 and a node in the output layer:

$\begin{matrix}{{O_{1}^{5} = {\sum\limits_{k}{O_{k}^{4}\left( {1 \leq k \leq 4} \right)}}},} & {{eq}.\mspace{14mu} 17}\end{matrix}$

In the training process, the engine tries to find the minimizing errorfunction between target value and the network output. For a giventraining data set with P entries, the error function is defined as:

$\begin{matrix}{{E = {{\sum\limits_{p = 1}^{P}E_{p}} = {\frac{1}{2}{\sum\limits_{p = 1}^{P}\left( {T_{p} - O_{1,p}^{5}} \right)^{2}}}}},\left( {1 \leq p \leq P} \right)} & {{eq}.\mspace{14mu} 18}\end{matrix}$

where O₁ ⁵ is the p^(th) output of the network and T_(p) is the p^(th)desired target. The premise parameters, {a_(ij),b_(ij)} are updatedaccording to a gradient descent and the consequent parameters {c_(k),d_(k)} are updated using a LMS algorithm.

The neural-fuzzy inference engine provides a combination of a fuzzyinference engine and an adaptive neural network. The neural-fuzzyinference engine uses fuzzy reasoning for both fuzzification anddefuzzification, that is, the membership functions may be monotonicnonlinear functions.

As described above, the neural-fuzzy inference engine can be applied tomulti-input and multi-output (MIMO) systems. For example, a system with20 inputs corresponding to brain activation within a portion of thebrain may provide two outputs corresponding to limbic control related toa flexor muscle and a extensor muscle. Various other embodiments mayinclude any number of inputs of brain activity and any number of outputscorresponding to limbic control.

The neural-fuzzy inference engine may use associated hybrid learningalgorithms to tune the parameters of membership functions such asfeedforward processes; least square estimation; backward process;gradient descent method, etc. The engine may also use an optimallearning rate that is updated after each learning process. Theneural-fuzzy inference engine may also use the least number ofcoefficients to learn and has a fast convergence rate.

The inference engine may integrate features of a fuzzy system (fuzzyreasoning) and neural networks (learning). Neural-fuzzy inferencetechnique may provide a means for fuzzy modeling to learn informationabout a data set, which will compute and generate the membershipfunction parameters, so that the associated fuzzy inference system cantrack the given input and output pattern. The inference engine'slearning method works similarly to that of neural networks. This networkcan be used to find out system parameters and unknown factors throughthe training process, which means it achieves the goal of systemidentification.

While this represents one mathematical approach in a five level process,it is to be understood that other mathematical variations and/or designscould be utilized in the inference engine. In addition to thisneural-fuzzy inference engine, signal processing may occur. For example,the data may be mean zeroed, time domain shifted, or filtered using aband-pass filter of any order, or may extract maximums and minimums,construct time domain file, remove leading and trailing data points,apply averages, resample data, apply noise reduction algorithms, etc. Asa specific example, the following eight-step signal processing may beperformed on the data prior to giving it to the neural-fuzzy system. Thefollowing highlights those eight steps: 1) obtain ASCII coded frequencysweep data files, 2) make data sets mean zeroed 3) apply 5^(th) orderband-pass filter, 4) overlap data files on a time domain, 5) extract themaximum value from each piece of band-passed and filtered data, andconstruct a one time domain data file, 6) remove the first 900 pointsand last 3000 data points, 7) apply a running average filter (withsumming every 50 data points), and 8) re-sample every 6^(th) order data.

III. Training System

Neural network training systems and methods are provided according toone embodiment of the invention. In order to train a neural network,inputs and corresponding outputs may be provided so that the weightingof each input can be established based on the known outputs. In the caseof training a brain imager, the data collected by the brain imager actsas the input signals while electromyography (EMG) data, for example,provides data for known outputs. EMG provides physiological responses ofmuscles at rest and during contraction. The training system correlatesEMG data with the brain activity data provided by the brain imager. Thiscorrelation may occur using a neural network and/or a neural-fuzzynetwork.

EMG units may be placed on the muscle group(s) of interest. An EMG unit(electromyograph) may detect the electrical potential generated bymuscle cells when in contraction or at rest. An EMG unit may measure themagnitude and/or frequency of the electric potential. A surface orneedle electrode may be used. Various other EMG units may be usedwithout deviating from the spirit of the invention.

Training a system for elbow and wrist actuation, for example, may usefour EMG units placed at the four major muscle groups that controlforward and reverse motion of the wrist joint. Activation of flexormuscles indicates a forward actuation of the corresponding joint andactivation of extensor muscles indicates a reverse actuation of thecorresponding joint. Accordingly, the EMG units are placed on the flexorand extensor muscles as needed.

During training, a brain imager is placed over the portion of a user'smotor cortex that controls the wrist. EMG units are placed on the wristextensor and flexor muscles. The user is then asked to move the wrist ina variety of ways. Brain activation data and EMG data are capturedduring the wrist motion. Data sets that correspond brain activation toEMG data may then be provided to a control system, such as a neuralnetwork. The data sets may be used by the neural network to adjust theneural network constants in such a way that the neural network providesthe outputs that correspond to the EMG data in response to inputs thatcorrespond to brain activation data.

By recording EMG and brain imaging data at the same frequency, matchingsets of inputs and outputs are provided to the fuzzy neural trainingsystem. After training has been completed, the EMG units can be removedfrom the system and an artificial limb may be controlled using thealgorithms developed by the neural network.

The neural network may also include feedforward and feedback controls.For example, brain activation signals are provided as feedforwardsignals and the EMG signals are feedback signals. The combination offeedforward and feedback signals may be used to train the neuralnetwork.

IV. Brain Imager and Prosthesis Control System

A brain imaging and prosthesis control system is also provided accordingto one embodiment of the invention. FIG. 11 shows a high-level blockdiagram of a prosthesis control system according to one embodiment ofthe invention. An optical brain imager 1100 images target tissues 1110.The target tissues may include portions of the motor cortex. The opticalbrain imager 1100 may provide signals from the brain to a neural-fuzzyinference engine 1120 that maps brain activation data to limbic controlsignals. The neural-fuzzy inference engine 1120 communicates limbiccontrol signals to a direct limb control system 1130 that controls anartificial limb.

FIG. 12 shows a block diagram of a brain imaging and prosthesis controlsystem according to another embodiment of the invention. A brain imager1000 includes a headset 1003 carrying the fiber optics, the LEDs, and/orthe photodiodes. The headset 1003 is connected to a control unit 1001which provides control signals 1002 to the headset, photodiodes, and/orLEDs and receives data signals back 1004. The LEDs may be substitutedwith other light sources, such as, for example, lasers. Moreover, thephotodiodes may be substituted with other light detectors.

Light source power may be controlled by adjusting the supply current.According to one embodiment of the invention, four LEDs are used thatare sequentially turned on and off. In such a configuration, only oneLED is on at a given time. The pulse duration of the LEDs may be lessthan 0.086 seconds. Other pulse durations may be used. The control unitmay comprise a compact unit that may be worn on the shoulder or upperback. The power unit 1010 may use any suitable power source such assolar power, rechargeable battery power, battery power, etc.

The brain imager 1000 is coupled with a controller 1015 that may includea microcontroller 1020 which may incorporate both the inference engine1021 and the direct limb control system 1023. The microcontroller 1020sends command signals 1023 to the drive mechanism 1030 of the artificiallimb 340 of the present invention to provide the signals that mayactuate the artificial limb 340. Such drive mechanisms 1030 for theartificial limbs are commercially available. Various other roboticand/or prosthetics may be used in place of an artificial limb.

The user of the optical brain imager 1000 of the present invention mayact as a portion of the feedback control loop for the artificial limbaccording to one embodiment of the invention. The user can see themovement of the limb and adjust limbic control accordingly. The feedbackthe user experiences may only provide visual confirmation of thelocation of the arm and any force or movement induced on the artificiallimb when it connects to the patient's body. As a result the brainchanges the degree of effort put into moving the limb to stop oraccelerate motion, which is in turn detected by the optical brain imager300 and leads to changes in the control signals sent to the limb.

In FIG. 12, the computer 1020 provides control signal 1024 to the brainimager control unit 1000 and receives data signals 1026 from it. Thedifference between the data streams 1024, 1026, 1002, and 1004 is thatthe data leaving the control unit has been processed (1002 and 1026). Inthe case of 1024 to 1002, the data has been modified from computersignals into a power. In the case of 1004 to 1026, the data has beenchanged from raw current measurements to computer-friendly information.The control unit performs the necessary amplifications and channelswitching to convert computer commands to LED control and dioderesponses back to computer data. The computer 1020 is also connected toreceive user inputs 1040 and to display results 1050. The computer maybe a microcomputer, processor, microprocessor, etc. The user inputs anddisplay results are both accomplished within an executable program thatallows textual control of power levels, displays the incoming data in agraphical format, and records incoming data to a file. However, in otherembodiments, these functions are embedded in a small microcontroller,e.g., a PC104 platform.

FIG. 13 shows another prosthesis control system using a brain imageraccording to one embodiment of the invention. Fiber optics 1115 channellight to and from the surface of a user's scalp using a headset 1105.Detectors record brain activity within the motor cortex. These signalsare provided to a neural-network 1120 which provides control signals fora prosthesis 1130.

Tele-operated devices controlled by neural commands could be used toprovide precise human manipulation in remote or hazardous environments,and neural controlled prosthetic limb could return function to aparalyzed patient by routing neural commands to actuators. Studies haveshown that patients who have already had a limb removed still exhibitactivation of the brain in the areas that correspond with the musclegroups of the missing limb. This phenomenon is referred to as “phantomlimb” and allows patients who no longer have muscle or nerve ending inthe vicinity of the missing limb to activate the optical brain imagersince their brain still attempts to send the signals. Thus, the outputof the neural-fuzzy inference engine and the brain imager 300 providelimbic control signals. One, two, three, four, five, six, seven, eight,nine, ten, eleven, or twelve signals, for example, may be provided for asingle joint or motion.

A flowchart outlining a method for providing prosthetic limbic controlfrom a brain imager is shown in FIG. 14 according to another embodimentof the invention. A portion of the brain is irradiated with NIR light atblock 1405. The portion of the brain irradiated with NIR light islocalized on a portion of the motor cortex of interest. Light from thebrain is detected at block 1410. The received light is converted intobrain activation data at block 1415. Photodiodes may be used to convertthe optical signals to brain activation data. As described above, byirradiating the motor cortex with light of two different wavelengths,one may determine the relative levels of oxy and deoxy-hemoglobin thatcorrespond to brain activity. The brain activity data is then providedto a neural network, for example, a fuzzy neural network at block 1420.Muscular outputs are then provided that correspond to the received brainactivity at block 1425. These muscular outputs may then be used tooperate or control a prosthesis.

V. Hair Penetrating Fiber Optic

FIGS. 15A and 15B show fiber optics that provide light to and detectlight from the scalp through hair according to one embodiment of theinvention. In FIG. 15A, the fiber optics 1505 has a distal tip thatincludes a bulb 1510. The bulb may include a flange and/or cone shapedmaterial. In FIG. 15B, the distal end of the fiber optic includes a ball1515. Both the ball 1515 and the bulb 1510 may be used to channel lightto and/or from the surface of a scalp through hair. The ball 1515 andbulb 1510 may be comprised of the same material as the fiber optic. Theball 1515 and bulb 1510 may be fused with the fiber optic. Such opticalfibers are capable of providing light to the surface of the scalp orreceiving light from the surface of the scalp through hairs.

VI. EEG Systems

In some embodiments of the invention the electrical activity can be usedinstead of or in conjunction with the optical activity used by the nearinfrared system described earlier. The electrical activity of the braincan be measured, for example, with an electroencephalograph (EEG)system. In some embodiments the integration of the temporal response ofthe EEG along with the spatial accuracy of near infrared systems canprovide an accurate noninvasive limbic control system. In otherembodiments, a system that detects electrical activity within the brain,such as an EEG, may be sufficient.

FIG. 16 shows a system that combines EEG signals 1605, EMG signals 1620and near IR signals 1615 for adaptive artificial limbic controlaccording to some embodiments of the invention. For example, using EMGsignals 1620 from healthy subjects, the adaptive control system can betrained to correlate EEG signals 1605 (or input patterns) and near IRsignals (or input patterns) 1615 to muscle activations. Once correlated,either or both of near IR signals 1615 and EMG signals 1620 can be usedby the adaptive control system 1610 to control artificial limb 1625.

EEG signals 1620 can be any type of electrical signal detected from thebrain. For example, an EEG system observes current passing through theneurons of during neuron activity. Often this current is very small. Andthis current can be referred to as the activation potential within theneurons. Electrodes placed on the surface of the brain and in differentregions can be used to detect and localize this current. The intensity,frequency, and localization of electrical current can be used toidentify portions of the brain associated with specific limbic activityand/or can be used in conjunction with the EMG signals for trainingpurposes. This training can create a relationship between brain activityand limbic control. Moreover, EEG systems can provide real-time brainactivity.

In some embodiments of the invention, a near infrared system can bepaired with an EEG system. For example, the array 210 shown in FIG. 2can include both near infrared sensors as well as EEG electrodes.Similarly, the array 310 shown in FIGS. 3A, 3B, and 3C can include IRsensors 350 and light sources 340 as shown, with electrodes interspersedbetween sensors 350 and light sources 340. This pairing of an EEG systemand a near infrared system can detect signals from the same or differentportions of the brain. And the two can be used together or singularly intraining and use.

FIG. 17 shows another high-level block diagram of a prosthesis controlsystem according to some embodiments of the invention. This prosthesiscontrol system is similar to the system shown in FIG. 11. But thissystem includes an EEG imager that senses electrical characteristics oftarget tissue 1110 and provides EEG signals to neural-fuzzy inferenceengine 1120. The optical brain imager 1100 may sense changes in theoptical characteristics of the brain and may provide signals from thebrain to a neural-fuzzy inference engine 1120. Neural-fuzzy inferenceengine 1120 can map the optical signals form optical brain imager 1100and EEG image to limbic control signals as described elsewhere withinthis disclosure. The neural-fuzzy inference engine 1120 communicateslimbic control signals to a direct limb control system 1130 thatcontrols an artificial limb. Target tissues 1110 may include portions ofthe motor cortex and the two imaging systems can target the same ordifferent portions of the brain.

FIG. 18 shows a flow chart of process 1800 for training a combined nearIR and EEG system according to some embodiments of the invention.Process 1800 can be used on a healthy patient to correlate (or map) EEGsignals and near IR signals with EMG signals. Process 1800 can includethree independent processes that monitor the brain and a limb during alimbic action: near IR signal collection, EEG signal collection, and EMGsignal collection. These three processes can occur in parallel or inseries. At block 1805, a portion of the brain can be irradiated withnear IR light. At block 1810 scattered light can be collected. Using thescattered light, an IR signal can be provided to the neural network atblock 1815. At block 1820 an electrical signal can be received from thebrain. The electrical signal, for example, can be recorded using an EEGsystem. The electrical (or EEG) signal can then be provided to theneural network. At block 1830 electrical signals from a muscle of musclegroup can be received. At block 1825, the EMG signals can be provided tothe neural network. At block 1840 the neural network can correlate (ormap) the near IR signals and the EMG signals with the EMG signal. Thus,using process 1800, limbic or muscular action can be associated withbrain activity.

The correlation of brain activity with limbic or muscle action can beused by a controller (e.g. adaptive control system 1610 in FIG. 16 orneuron-fuzzy inference engine 1120 in FIG. 11 and/or FIG. 17) to controlan artificial limb when receiving a EEG signals and near IR signals.Process 1900 shown in FIG. 19 is an example of process that can be usedby the controller for this purpose. At block 1905 EEG signals can bereceived from an EEG system coupled with a user's brain. At block 1910IR signals can be received from an IR system coupled with the user'sbrain. These two signals can be recorded in parallel or series. In someembodiments, the EEG signal may have a faster response time than the IRsignal and may be received first. At block 1915 a muscular output can bedetermined based on a correlation between EEG and IR signals with EMGsignals as determined in process 1800. At block 1920 a prosthesis can beactuated accordingly.

VII. Lie Detectors

The systems or methods described in the various embodiments can be usedfor other purposes. For example, a brain imaging system can be used todetect the truth or falsehood of a statement. FIG. 20 shows a flowchartof process 2000 using brain imaging for a lie detection. At block 2005,a subject can be asked to provide a false statement and a false answercan be received. The question, for example, can be a question that hasan unambiguously true or false answer such as asking for the subject'sbirth date, name, etc. Questions with yes or no answers can be presentedthat are patently true or false. The answer can also be verified asbeing false. Signals from a brain imager, such as IR signals or EEGsignals or both, can be received and noted as being associated with afalse statement at block 2010. At block 2015, the subject can be askedto provide a true statement and a true answer can be received. Thequestion, for example, can be a question that has an unambiguously trueor false answer such as asking for the subject's birth date, name, etc.Questions with yes or no answers can be presented that are patently trueor false. The answer can be verified as being true. Signals from a brainimager, such as IR signals or EEG signals or both, can be received andnoted as being associated with a true statement at block 2020.

At block 2025, the imaging signals and can be correlated (or mapped)with the true or false statements. This correlation can include usingvarious neural networks to train the system. Or the correlation canoccur using the neural-fuzzy inference engine described above. Moreover,the imaging signals can be correlated using various embodimentsdiscussed above in regard to limbic control. Blocks 2005, 2010, 2015,and 2020 can be repeated as needed to train the system. As theneuron-network matures the process can predict whether the statementsare true or false. After a certain number of correct predictions thesystem can be determined to be trained.

At block 2030 the subject can be asked a question without a verifiabletrue or false answer. And an answer can be received. Brain imagingsignals (e.g. IR and/or EEG signals) can be received at block 2035 and apredication can be made about the veracity of the answer based on thereceived brain imaging signals. Thus, brain imaging techniques describedin conjunction with the limbic control system can be implemented in alie detection system.

Specific details are given in the above description to provide athorough understanding of the embodiments. However, it is understoodthat the embodiments may be practiced without these specific details.For example, circuits may be shown in block diagrams in order not toobscure the embodiments in unnecessary detail. In other instances,well-known circuits, processes, algorithms, structures, and techniquesmay be shown without unnecessary detail in order to avoid obscuring theembodiments.

Implementation of the techniques, blocks, steps and means describedabove may be done in various ways. For example, these techniques,blocks, steps and means may be implemented in hardware, software, or acombination thereof. For a hardware implementation, the processing unitsmay be implemented within one or more application specific integratedcircuits (ASICs), digital signal processors (DSPs), digital signalprocessing devices (DSPDs), programmable logic devices (PLDs), fieldprogrammable gate arrays (FPGAs), processors, controllers,micro-controllers, microprocessors, and other electronic units designedto perform the functions described above and/or a combination thereof.

Also, it is noted that the embodiments may be described as a processwhich is depicted as a flowchart, a flow diagram, a data flow diagram, astructure diagram, or a block diagram. Although a flowchart may describethe operations as a sequential process, many of the operations can beperformed in parallel or concurrently. In addition, the order of theoperations may be rearranged. A process is terminated when itsoperations are completed, but could have additional steps not includedin the figure. A process may correspond to a method, a function, aprocedure, a subroutine, a subprogram, etc. When a process correspondsto a function, its termination corresponds to a return of the functionto the calling function or the main function.

Furthermore, embodiments may be implemented by hardware, software,scripting languages, firmware, middleware, microcode, hardwaredescription languages and/or any combination thereof. When implementedin software, firmware, middleware, scripting language and/or microcode,the program code or code segments to perform the necessary tasks may bestored in a machine readable medium, such as a storage medium. A codesegment or machine-executable instruction may represent a procedure, afunction, a subprogram, a program, a routine, a subroutine, a module, asoftware package, a script, a class, or any combination of instructions,data structures and/or program statements. A code segment may be coupledto another code segment or a hardware circuit by passing and/orreceiving information, data, arguments, parameters and/or memorycontents. Information, arguments, parameters, data, etc. may be passed,forwarded, or transmitted via any suitable means including memorysharing, message passing, token passing, network transmission, etc.

For a firmware and/or software implementation, the methodologies may beimplemented with modules (e.g., procedures, functions, and so on) thatperform the functions described herein. Any machine-readable mediumtangibly embodying instructions may be used in implementing themethodologies described herein. For example, software codes may bestored in a memory. Memory may be implemented within the processor orexternal to the processor. As used herein the term “memory” refers toany type of long term, short term, volatile, nonvolatile, or otherstorage medium and is not to be limited to any particular type of memoryor number of memories, or type of media upon which memory is stored.

Moreover, as disclosed herein, the term “storage medium” may representone or more devices for storing data, including read only memory (ROM),random access memory (RAM), magnetic RAM, core memory, magnetic diskstorage mediums, optical storage mediums, flash memory devices and/orother machine readable mediums for storing information. The term“machine-readable medium” includes, but is not limited to portable orfixed storage devices, optical storage devices, wireless channels and/orvarious other mediums capable of storing, containing or carryinginstruction(s) and/or data.

While the principles of the disclosure have been described above inconnection with specific apparatuses and methods, it is to be clearlyunderstood that this description is made only by way of example and notas limitation on the scope of the disclosure.

1. A method for controlling a prosthesis, the method comprising:receiving two or more input signals from at least two distinct brainimagers, wherein the input signals corresponds to brain activity at aportion of the motor cortex; using a neural network to map the inputsignals to an output signal, wherein the neural network is trained tomap the input signals associated with the portion of the motor cortex toan output signal that corresponds with a muscle group; and coupling theoutput signal with the prosthesis, wherein the prosthesis is configuredto respond to the output signal.
 2. The method according to claim 1,wherein the neural network comprises a fuzzy neural network.
 3. Themethod according to claim 1, wherein one of the distinct brain imagerscomprises an electrical imaging system, and the method further comprisesdetecting action potentials from neurons within the portion of the motorcortex.
 4. The method according to claim 2, wherein the electricalimaging system comprises an EEG system.
 5. The method according to claim1, wherein one of the distinct brain imagers comprises an opticalimager, and the method further comprises: illuminating light on aportion of the brain using a light source; and receiving a light signalat the scalp using a photodiode, wherein the detected light travels fromthe light source through a sub-portion of the brain and is detected atthe photodiode.
 6. The method according to claim 5, wherein the lightsource illuminates a portion of the brain with near infrared light. 7.The method according to claim 5, wherein the one or more light sourcescomprise at least one or more LEDs and one or more lasers.
 8. Aprosthesis control system, comprising: one or more light sourcesconfigured to irradiate light into a first portion of the brain; one ormore photodiodes configured to detect a portion of the light transmittedinto the first portion of the brain, wherein the detected light travelsat least from the one or more light sources through a plurality ofsub-portions of the brain and is detected at the plurality ofphotodiodes; one or more electrodes configured to detect actionpotentials from neurons within a second portion of the brain; and acontroller coupled with the one or more photodiodes, the one or moreelectrodes, and the prosthesis, wherein the controller is configured toreceive a plurality of inputs from the plurality of photodiodes and theone or more electrodes, wherein the controller is configured to:determine the brain activity at a plurality of sub-portions of the firstportion of the brain from the input from the plurality of photodiodes;determine the brain activity at a plurality of sub-portions of thesecond portion of the brain from the inputs from the electrodes; anddetermine a plurality of limbic control signals from the brain activitywithin the first portion and the second portion of the brain.
 9. Theprosthesis control system according to claim 8 further comprising aprosthesis coupled with the controller and wherein the controller isfurther configured to send the plurality of limbic control signals tothe prosthesis.
 10. The prosthesis control system according to claim 8,wherein the electrodes are Electroencephalograph electrodes.
 11. Theprosthesis control system according to claim 8, wherein the determiningthe brain activity at a plurality of sub-portions of the first portionof the brain further comprises: determine the relative concentration ofoxy-hemoglobin within the first portion of the brain from the pluralityof photodiodes inputs; determine the relative concentration ofhemoglobin within the first portion of the brain from the plurality ofphotodiodes inputs; and determine the brain activity at a plurality ofsub-portions of the first portion of the brain from the relativeconcentrations of oxy-hemoglobin and hemoglobin.
 12. The prosthesiscontrol system according to claim 8, wherein the first and secondportions of the brain comprise the same portion of the brain.
 13. Theprosthesis control system according to claim 8, wherein the controllercomprises a neural network inference engine configured to determine theplurality of limbic control signals from the brain activity within thefirst portion of the brain.
 14. The prosthesis control system accordingto claim 8, wherein the controller comprises a neural network.
 15. Theprosthesis control system according to claim 8, wherein the controllercomprises a fuzzy neural network.
 16. The prosthesis control systemaccording to claim 8 comprising a muscle activity sensor, wherein thecontroller utilizes brain activity at the plurality of sub-portions ofthe first portion of the brain as feedforward signals and muscleactivation as recorded by the muscle activity sensor as feedbacksignals.
 17. A method for training a prosthesis system, wherein theprosthesis system comprises a neural network, a brain imaging system,and a prosthesis, the training utilizing a muscle activity detector, themethod comprising: receiving brain activity data from the brain imagingsystem; receiving muscular response data from the muscle activitydetector, wherein the muscular response data corresponds with the brainactivity; and training the neural network to produce the muscularresponse data from the brain activity data.
 18. The method according toclaim 17, wherein the neural network is a fuzzy neural network.
 19. Themethod according to claim 17, wherein the muscle activity detectorcomprises an electromyograph.
 20. The method according to claim 17,wherein the brain imaging system comprises both an IR imaging system andan EEG system.